Orientation-Based Playback Device Microphone Selection

ABSTRACT

Aspects of a multi-orientation playback device including at least one microphone array are discussed. A method may include determining an orientation of the playback device which includes at least one microphone array and determining at least one microphone training response for the playback device from a plurality of microphone training responses based on the orientation of the playback device. The at least one microphone array can detect a sound input, and the location information of a source of the sound input can be determined based on the at least one microphone training response and the detected sound input. Based on the location information of the source, the directional focus of the at least one microphone array can be adjusted, and the sound input can be captured based on the adjusted directional focus.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority under 35 U.S.C. § 120 to, and is acontinuation of, U.S. patent application Ser. No. 15/681,937 filed Aug.21, 2017 and entitled “Multi-Orientation Playback Device Microphones”,which is a continuation of U.S. patent application Ser. No. 15/282,554filed Sep. 30, 2016 and entitled “Multi-Orientation Playback DeviceMicrophones”, the contents of each of which are herein incorporated byreference in their entirety for all purposes.

FIELD OF THE DISCLOSURE

The disclosure is related to consumer goods and, more particularly, tomethods, systems, products, features, services, and other elementsdirected to media playback or some aspect thereof.

BACKGROUND

Options for accessing and listening to digital audio in an out-loudsetting were limited until in 2003, when SONOS, Inc. filed for one ofits first patent applications, entitled “Method for Synchronizing AudioPlayback between Multiple Networked Devices,” and began offering a mediaplayback system for sale in 2005. The Sonos Wireless HiFi System enablespeople to experience music from many sources via one or more networkedplayback devices. Through a software control application installed on asmartphone, tablet, or computer, one can play what he or she wants inany room that has a networked playback device. Additionally, using thecontroller, for example, different songs can be streamed to each roomwith a playback device, rooms can be grouped together for synchronousplayback, or the same song can be heard in all rooms synchronously.

Given the ever growing interest in digital media, there continues to bea need to develop consumer-accessible technologies to further enhancethe listening experience.

BRIEF DESCRIPTION OF THE DRAWINGS

Features, aspects, and advantages of the presently disclosed technologymay be better understood with regard to the following description,appended claims, and accompanying drawings where:

FIG. 1 shows an example media playback system configuration in whichcertain embodiments may be practiced;

FIG. 2 shows a functional block diagram of an example playback deviceaccording to aspects described herein;

FIG. 3 shows a functional block diagram of an example control deviceaccording to aspects described herein;

FIG. 4 shows an example controller interface according to aspectsdescribed herein;

FIG. 5 shows an example plurality of network devices according toaspects described herein;

FIG. 6 shows a function block diagram of an example network microphonedevice according to aspects described herein;

FIGS. 7A-7B depict respective perspective views of orientations of aplayback device according to aspects described herein;

FIG. 8 illustrates an example flow diagram 800 of a method forprocessing sound input based on playback device orientation according toaspects described herein; and

FIG. 9 shows a top view of an example calibration setup according toaspects described herein.

The drawings are for the purpose of illustrating example embodiments,but it is understood that the inventions are not limited to thearrangements and instrumentality shown in the drawings.

DETAILED DESCRIPTION I. Overview

Some embodiments described herein involve adjusting directional focus ofat least one microphone array trained based on the orientation of aplayback device. Microphone arrays on a multi-orientation device mayhave different sensitivities and response profiles to sound coming fromthe same direction across different orientations of the device. Inobtaining training response values (e.g., calibration response values,measured response values) of the microphone arrays in different playbackdevice orientations, a general response profile for various playbackdevice orientations can be created. In comparing the general responseprofile to a response profile of a received sound input response, alocation (e.g., direction in one or more dimensions) of the source ofthe sound input can be identified or estimated.

The examples provided herein involve a method, a playback device, and asystem. The method may include determining an orientation of theplayback device which includes at least one microphone array anddetermining at least one microphone training response for the playbackdevice from a plurality of microphone training responses based on theorientation of the playback device. The at least one microphone arraycan detect a sound input, and the location information of a source ofthe sound input can be determined based on the at least one microphonetraining response and the detected sound input. Based on the locationinformation of the source, the directional focus of the at least onemicrophone array can be adjusted, and the sound input can be capturedbased on the adjusted directional focus.

In another aspect, a non-transitory computer-readable medium isprovided. The non-transitory computer readable medium has stored thereoninstructions executable by a computing device to cause the computingdevice to perform functions. The functions include determining anorientation of the playback device which includes at least onemicrophone array and determining at least one microphone trainingresponse for the playback device from a plurality of microphone trainingresponses based on the orientation of the playback device. The at leastone microphone array can detect a sound input, and the locationinformation of a source of the sound input can be determined based onthe at least one microphone training response and the detected soundinput. Based on the location information of the source, the directionalfocus of the at least one microphone array can be adjusted, and thesound input can be captured based on the adjusted directional focus.

In yet another aspect, an apparatus is provided. The apparatus includesa processor and a memory. The memory has stored thereon instructionsexecutable by the apparatus to cause the system to perform functions.The functions include determining an orientation of the playback devicewhich includes at least one microphone array and determining at leastone microphone training response for the playback device from aplurality of microphone training responses based on the orientation ofthe playback device. The at least one microphone array can detect asound input, and the location information of a source of the sound inputcan be determined based on the at least one microphone training responseand the detected sound input. Based on the location information of thesource, the directional focus of the at least one microphone array canbe adjusted, and the sound input can be captured based on the adjusteddirectional focus.

While some examples described herein may refer to functions performed bygiven actors such as “users” and/or other entities, it should beunderstood that this is for purposes of explanation only. The claimsshould not be interpreted to require action by any such example actorunless explicitly required by the language of the claims themselves. Itwill be understood by one of ordinary skill in the art that thisdisclosure includes numerous other embodiments.

II. Example Operating Environment

FIG. 1 shows an example configuration of a media playback system 100 inwhich one or more embodiments disclosed herein may be practiced orimplemented. The media playback system 100 as shown is associated withan example home environment having several rooms and spaces, such as forexample, a master bedroom, an office, a dining room, and a living room.As shown in the example of FIG. 1, the media playback system 100includes playback devices 102-124, control devices 126 and 128, and awired or wireless network router 130.

Further discussions relating to the different components of the examplemedia playback system 100 and how the different components may interactto provide a user with a media experience may be found in the followingsections. While discussions herein may generally refer to the examplemedia playback system 100, technologies described herein are not limitedto applications within, among other things, the home environment asshown in FIG. 1. For instance, the technologies described herein may beuseful in environments where multi-zone audio may be desired, such as,for example, a commercial setting like a restaurant, mall or airport, avehicle like a sports utility vehicle (SUV), bus or car, a ship or boat,an airplane, and so on.

a. Example Playback Devices

FIG. 2 shows a functional block diagram of an example playback device200 that may be configured to be one or more of the playback devices102-124 of the media playback system 100 of FIG. 1. The playback device200 may include a processor 202, software components 204, memory 206,audio processing components 208, audio amplifier(s) 210, speaker(s) 212,a network interface 214 including wireless interface(s) 216 and wiredinterface(s) 218, and microphone(s) 220. In one case, the playbackdevice 200 may not include the speaker(s) 212, but rather a speakerinterface for connecting the playback device 200 to external speakers.In another case, the playback device 200 may include neither thespeaker(s) 212 nor the audio amplifier(s) 210, but rather an audiointerface for connecting the playback device 200 to an external audioamplifier or audio-visual receiver.

In one example, the processor 202 may be a clock-driven computingcomponent configured to process input data according to instructionsstored in the memory 206. The memory 206 may be a tangiblecomputer-readable medium configured to store instructions executable bythe processor 202. For instance, the memory 206 may be data storage thatcan be loaded with one or more of the software components 204 executableby the processor 202 to achieve certain functions. In one example, thefunctions may involve the playback device 200 retrieving audio data froman audio source or another playback device. In another example, thefunctions may involve the playback device 200 sending audio data toanother device or playback device on a network. In yet another example,the functions may involve pairing of the playback device 200 with one ormore playback devices to create a multi-channel audio environment.

Certain functions may involve the playback device 200 synchronizingplayback of audio content with one or more other playback devices.During synchronous playback, a listener will preferably not be able toperceive time-delay differences between playback of the audio content bythe playback device 200 and the one or more other playback devices. U.S.Pat. No. 8,234,395 entitled, “System and method for synchronizingoperations among a plurality of independently clocked digital dataprocessing devices,” which is hereby incorporated by reference, providesin more detail some examples for audio playback synchronization amongplayback devices.

The memory 206 may further be configured to store data associated withthe playback device 200, such as one or more zones and/or zone groupsthe playback device 200 is a part of, audio sources accessible by theplayback device 200, or a playback queue that the playback device 200(or some other playback device) may be associated with. The data may bestored as one or more state variables that are periodically updated andused to describe the state of the playback device 200. The memory 206may also include the data associated with the state of the other devicesof the media system, and shared from time to time among the devices sothat one or more of the devices have the most recent data associatedwith the system. Other embodiments are also possible.

The audio processing components 208 may include one or moredigital-to-analog converters (DAC), an audio preprocessing component, anaudio enhancement component or a digital signal processor (DSP), and soon. In one embodiment, one or more of the audio processing components208 may be a subcomponent of the processor 202. In one example, audiocontent may be processed and/or intentionally altered by the audioprocessing components 208 to produce audio signals. The produced audiosignals may then be provided to the audio amplifier(s) 210 foramplification and playback through speaker(s) 212. Particularly, theaudio amplifier(s) 210 may include devices configured to amplify audiosignals to a level for driving one or more of the speakers 212. Thespeaker(s) 212 may include an individual transducer (e.g., a “driver”)or a complete speaker system involving an enclosure with one or moredrivers. A particular driver of the speaker(s) 212 may include, forexample, a subwoofer (e.g., for low frequencies), a mid-range driver(e.g., for middle frequencies), and/or a tweeter (e.g., for highfrequencies). In some cases, each transducer in the one or more speakers212 may be driven by an individual corresponding audio amplifier of theaudio amplifier(s) 210. In addition to producing analog signals forplayback by the playback device 200, the audio processing components 208may be configured to process audio content to be sent to one or moreother playback devices for playback.

Audio content to be processed and/or played back by the playback device200 may be received from an external source, such as via an audioline-in input connection (e.g., an auto-detecting 3.5 mm audio line-inconnection) or the network interface 214.

The network interface 214 may be configured to facilitate a data flowbetween the playback device 200 and one or more other devices on a datanetwork. As such, the playback device 200 may be configured to receiveaudio content over the data network from one or more other playbackdevices in communication with the playback device 200, network deviceswithin a local area network, or audio content sources over a wide areanetwork such as the Internet. In one example, the audio content andother signals transmitted and received by the playback device 200 may betransmitted in the form of digital packet data containing an InternetProtocol (IP)-based source address and IP-based destination addresses.In such a case, the network interface 214 may be configured to parse thedigital packet data such that the data destined for the playback device200 is properly received and processed by the playback device 200.

As shown, the network interface 214 may include wireless interface(s)216 and wired interface(s) 218. The wireless interface(s) 216 mayprovide network interface functions for the playback device 200 towirelessly communicate with other devices (e.g., other playbackdevice(s), speaker(s), receiver(s), network device(s), control device(s)within a data network the playback device 200 is associated with) inaccordance with a communication protocol (e.g., any wireless standardincluding IEEE 802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 802.15, 4Gmobile communication standard, and so on). The wired interface(s) 218may provide network interface functions for the playback device 200 tocommunicate over a wired connection with other devices in accordancewith a communication protocol (e.g., IEEE 802.3). While the networkinterface 214 shown in FIG. 2 includes both wireless interface(s) 216and wired interface(s) 218, the network interface 214 may in someembodiments include only wireless interface(s) or only wiredinterface(s).

The microphone(s) 220 may be arranged to detect sound in the environmentof the playback device 200. For instance, the microphone(s) may bemounted on an exterior wall of a housing of the playback device. Themicrophone(s) may be any type of microphone now known or later developedsuch as a condenser microphone, electret condenser microphone, or adynamic microphone. The microphone(s) may be sensitive to a portion ofthe frequency range of the speaker(s) 220. One or more of the speaker(s)220 may operate in reverse as the microphone(s) 220. In some aspects,the playback device 200 might not have microphone(s) 220.

In one example, the playback device 200 and one other playback devicemay be paired to play two separate audio components of audio content.For instance, playback device 200 may be configured to play a leftchannel audio component, while the other playback device may beconfigured to play a right channel audio component, thereby producing orenhancing a stereo effect of the audio content. The paired playbackdevices (also referred to as “bonded playback devices”) may further playaudio content in synchrony with other playback devices.

In another example, the playback device 200 may be sonicallyconsolidated with one or more other playback devices to form a single,consolidated playback device. A consolidated playback device may beconfigured to process and reproduce sound differently than anunconsolidated playback device or playback devices that are paired,because a consolidated playback device may have additional speakerdrivers through which audio content may be rendered. For instance, ifthe playback device 200 is a playback device designed to render lowfrequency range audio content (i.e. a subwoofer), the playback device200 may be consolidated with a playback device designed to render fullfrequency range audio content. In such a case, the full frequency rangeplayback device, when consolidated with the low frequency playbackdevice 200, may be configured to render only the mid and high frequencycomponents of audio content, while the low frequency range playbackdevice 200 renders the low frequency component of the audio content. Theconsolidated playback device may further be paired with a singleplayback device or yet another consolidated playback device.

By way of illustration, SONOS, Inc. presently offers (or has offered)for sale certain playback devices including a “PLAY:1,” “PLAY:3,”“PLAY:5,” “PLAYBAR,” “CONNECT:AMP,” “CONNECT,” and “SUB.” Any otherpast, present, and/or future playback devices may additionally oralternatively be used to implement the playback devices of exampleembodiments disclosed herein. Additionally, it is understood that aplayback device is not limited to the example illustrated in FIG. 2 orto the SONOS product offerings. For example, a playback device mayinclude a wired or wireless headphone. In another example, a playbackdevice may include or interact with a docking station for personalmobile media playback devices. In yet another example, a playback devicemay be integral to another device or component such as a television, alighting fixture, or some other device for indoor or outdoor use.

b. Example Playback Zone Configurations

Referring back to the media playback system 100 of FIG. 1, theenvironment may have one or more playback zones, each with one or moreplayback devices. The media playback system 100 may be established withone or more playback zones, after which one or more zones may be added,or removed to arrive at the example configuration shown in FIG. 1. Eachzone may be given a name according to a different room or space such asan office, bathroom, master bedroom, bedroom, kitchen, dining room,living room, and/or balcony. In one case, a single playback zone mayinclude multiple rooms or spaces. In another case, a single room orspace may include multiple playback zones.

As shown in FIG. 1, the balcony, dining room, kitchen, bathroom, office,and bedroom zones each have one playback device, while the living roomand master bedroom zones each have multiple playback devices. In theliving room zone, playback devices 104, 106, 108, and 110 may beconfigured to play audio content in synchrony as individual playbackdevices, as one or more bonded playback devices, as one or moreconsolidated playback devices, or any combination thereof. Similarly, inthe case of the master bedroom, playback devices 122 and 124 may beconfigured to play audio content in synchrony as individual playbackdevices, as a bonded playback device, or as a consolidated playbackdevice.

In one example, one or more playback zones in the environment of FIG. 1may each be playing different audio content. For instance, the user maybe grilling in the balcony zone and listening to hip hop music beingplayed by the playback device 102 while another user may be preparingfood in the kitchen zone and listening to classical music being playedby the playback device 114. In another example, a playback zone may playthe same audio content in synchrony with another playback zone. Forinstance, the user may be in the office zone where the playback device118 is playing the same rock music that is being playing by playbackdevice 102 in the balcony zone. In such a case, playback devices 102 and118 may be playing the rock music in synchrony such that the user mayseamlessly (or at least substantially seamlessly) enjoy the audiocontent that is being played out-loud while moving between differentplayback zones. Synchronization among playback zones may be achieved ina manner similar to that of synchronization among playback devices, asdescribed in previously referenced U.S. Pat. No. 8,234,395.

As suggested above, the zone configurations of the media playback system100 may be dynamically modified, and in some embodiments, the mediaplayback system 100 supports numerous configurations. For instance, if auser physically moves one or more playback devices to or from a zone,the media playback system 100 may be reconfigured to accommodate thechange(s). For instance, if the user physically moves the playbackdevice 102 from the balcony zone to the office zone, the office zone maynow include both the playback device 118 and the playback device 102.The playback device 102 may be paired or grouped with the office zoneand/or renamed if so desired via a control device such as the controldevices 126 and 128. On the other hand, if the one or more playbackdevices are moved to a particular area in the home environment that isnot already a playback zone, a new playback zone may be created for theparticular area.

Further, different playback zones of the media playback system 100 maybe dynamically combined into zone groups or split up into individualplayback zones. For instance, the dining room zone and the kitchen zone114 may be combined into a zone group for a dinner party such thatplayback devices 112 and 114 may render audio content in synchrony. Onthe other hand, the living room zone may be split into a television zoneincluding playback device 104, and a listening zone including playbackdevices 106, 108, and 110, if the user wishes to listen to music in theliving room space while another user wishes to watch television.

c. Example Control Devices

FIG. 3 shows a functional block diagram of an example control device 300that may be configured to be one or both of the control devices 126 and128 of the media playback system 100. As shown, the control device 300may include a processor 302, memory 304, a network interface 306, a userinterface 308, microphone(s) 310, and software components 312. In oneexample, the control device 300 may be a dedicated controller for themedia playback system 100. In another example, the control device 300may be a network device on which media playback system controllerapplication software may be installed, such as for example, an iPhone7iPad™ or any other smart phone, tablet or network device (e.g., anetworked computer such as a PC or Mac™).

The processor 302 may be configured to perform functions relevant tofacilitating user access, control, and configuration of the mediaplayback system 100. The memory 304 may be data storage that can beloaded with one or more of the software components executable by theprocessor 302 to perform those functions. The memory 304 may also beconfigured to store the media playback system controller applicationsoftware and other data associated with the media playback system 100and the user.

In one example, the network interface 306 may be based on an industrystandard (e.g., infrared, radio, wired standards including IEEE 802.3,wireless standards including IEEE 802.11a, 802.11b, 802.11g, 802.11n,802.11ac, 802.15, 4G mobile communication standard, and so on). Thenetwork interface 306 may provide a means for the control device 300 tocommunicate with other devices in the media playback system 100. In oneexample, data and information (e.g., such as a state variable) may becommunicated between control device 300 and other devices via thenetwork interface 306. For instance, playback zone and zone groupconfigurations in the media playback system 100 may be received by thecontrol device 300 from a playback device or another network device, ortransmitted by the control device 300 to another playback device ornetwork device via the network interface 306. In some cases, the othernetwork device may be another control device.

Playback device control commands such as volume control and audioplayback control may also be communicated from the control device 300 toa playback device via the network interface 306. As suggested above,changes to configurations of the media playback system 100 may also beperformed by a user using the control device 300. The configurationchanges may include adding/removing one or more playback devices to/froma zone, adding/removing one or more zones to/from a zone group, forminga bonded or consolidated player, separating one or more playback devicesfrom a bonded or consolidated player, among others. Accordingly, thecontrol device 300 may sometimes be referred to as a controller, whetherthe control device 300 is a dedicated controller or a network device onwhich media playback system controller application software isinstalled.

Control device 300 may include microphone(s) 310. Microphone(s) 310 maybe arranged to detect sound in the environment of the control device300. Microphone(s) 310 may be any type of microphone now known or laterdeveloped such as a condenser microphone, electret condenser microphone,or a dynamic microphone. The microphone(s) may be sensitive to a portionof a frequency range. Two or more microphones 310 may be arranged tocapture location information of an audio source (e.g., voice, audiblesound) and/or to assist in filtering background noise.

The user interface 308 of the control device 300 may be configured tofacilitate user access and control of the media playback system 100, byproviding a controller interface such as the controller interface 400shown in FIG. 4. The controller interface 400 includes a playbackcontrol region 410, a playback zone region 420, a playback status region430, a playback queue region 440, and an audio content sources region450. The user interface 400 as shown is just one example of a userinterface that may be provided on a network device such as the controldevice 300 of FIG. 3 (and/or the control devices 126 and 128 of FIG. 1)and accessed by users to control a media playback system such as themedia playback system 100. Other user interfaces of varying formats,styles, and interactive sequences may alternatively be implemented onone or more network devices to provide comparable control access to amedia playback system.

The playback control region 410 may include selectable (e.g., by way oftouch or by using a cursor) icons to cause playback devices in aselected playback zone or zone group to play or pause, fast forward,rewind, skip to next, skip to previous, enter/exit shuffle mode,enter/exit repeat mode, enter/exit cross fade mode. The playback controlregion 410 may also include selectable icons to modify equalizationsettings, and playback volume, among other possibilities.

The playback zone region 420 may include representations of playbackzones within the media playback system 100. In some embodiments, thegraphical representations of playback zones may be selectable to bringup additional selectable icons to manage or configure the playback zonesin the media playback system, such as a creation of bonded zones,creation of zone groups, separation of zone groups, and renaming of zonegroups, among other possibilities.

For example, as shown, a “group” icon may be provided within each of thegraphical representations of playback zones. The “group” icon providedwithin a graphical representation of a particular zone may be selectableto bring up options to select one or more other zones in the mediaplayback system to be grouped with the particular zone. Once grouped,playback devices in the zones that have been grouped with the particularzone will be configured to play audio content in synchrony with theplayback device(s) in the particular zone. Analogously, a “group” iconmay be provided within a graphical representation of a zone group. Inthis case, the “group” icon may be selectable to bring up options todeselect one or more zones in the zone group to be removed from the zonegroup. Other interactions and implementations for grouping andungrouping zones via a user interface such as the user interface 400 arealso possible. The representations of playback zones in the playbackzone region 420 may be dynamically updated as playback zone or zonegroup configurations are modified.

The playback status region 430 may include graphical representations ofaudio content that is presently being played, previously played, orscheduled to play next in the selected playback zone or zone group. Theselected playback zone or zone group may be visually distinguished onthe user interface, such as within the playback zone region 420 and/orthe playback status region 430. The graphical representations mayinclude track title, artist name, album name, album year, track length,and other relevant information that may be useful for the user to knowwhen controlling the media playback system via the user interface 400.

The playback queue region 440 may include graphical representations ofaudio content in a playback queue associated with the selected playbackzone or zone group. In some embodiments, each playback zone or zonegroup may be associated with a playback queue containing informationcorresponding to zero or more audio items for playback by the playbackzone or zone group. For instance, each audio item in the playback queuemay comprise a uniform resource identifier (URI), a uniform resourcelocator (URL) or some other identifier that may be used by a playbackdevice in the playback zone or zone group to find and/or retrieve theaudio item from a local audio content source or a networked audiocontent source, possibly for playback by the playback device.

In one example, a playlist may be added to a playback queue, in whichcase information corresponding to each audio item in the playlist may beadded to the playback queue. In another example, audio items in aplayback queue may be saved as a playlist. In a further example, aplayback queue may be empty, or populated but “not in use” when theplayback zone or zone group is playing continuously streaming audiocontent, such as Internet radio that may continue to play untilotherwise stopped, rather than discrete audio items that have playbackdurations. In an alternative embodiment, a playback queue can includeInternet radio and/or other streaming audio content items and be “inuse” when the playback zone or zone group is playing those items. Otherexamples are also possible.

When playback zones or zone groups are “grouped” or “ungrouped,”playback queues associated with the affected playback zones or zonegroups may be cleared or re-associated. For example, if a first playbackzone including a first playback queue is grouped with a second playbackzone including a second playback queue, the established zone group mayhave an associated playback queue that is initially empty, that containsaudio items from the first playback queue (such as if the secondplayback zone was added to the first playback zone), that contains audioitems from the second playback queue (such as if the first playback zonewas added to the second playback zone), or a combination of audio itemsfrom both the first and second playback queues. Subsequently, if theestablished zone group is ungrouped, the resulting first playback zonemay be re-associated with the previous first playback queue, or beassociated with a new playback queue that is empty or contains audioitems from the playback queue associated with the established zone groupbefore the established zone group was ungrouped. Similarly, theresulting second playback zone may be re-associated with the previoussecond playback queue, or be associated with a new playback queue thatis empty, or contains audio items from the playback queue associatedwith the established zone group before the established zone group wasungrouped. Other examples are also possible.

Referring back to the user interface 400 of FIG. 4, the graphicalrepresentations of audio content in the playback queue region 440 mayinclude track titles, artist names, track lengths, and other relevantinformation associated with the audio content in the playback queue. Inone example, graphical representations of audio content may beselectable to bring up additional selectable icons to manage and/ormanipulate the playback queue and/or audio content represented in theplayback queue. For instance, a represented audio content may be removedfrom the playback queue, moved to a different position within theplayback queue, or selected to be played immediately, or after anycurrently playing audio content, among other possibilities. A playbackqueue associated with a playback zone or zone group may be stored in amemory on one or more playback devices in the playback zone or zonegroup, on a playback device that is not in the playback zone or zonegroup, and/or some other designated device.

The audio content sources region 450 may include graphicalrepresentations of selectable audio content sources from which audiocontent may be retrieved and played by the selected playback zone orzone group. Discussions pertaining to audio content sources may be foundin the following section.

d. Example Audio Content Sources

As indicated previously, one or more playback devices in a zone or zonegroup may be configured to retrieve for playback audio content (e.g.according to a corresponding URI or URL for the audio content) from avariety of available audio content sources. In one example, audiocontent may be retrieved by a playback device directly from acorresponding audio content source (e.g., a line-in connection). Inanother example, audio content may be provided to a playback device overa network via one or more other playback devices or network devices.

Example audio content sources may include a memory of one or moreplayback devices in a media playback system such as the media playbacksystem 100 of FIG. 1, local music libraries on one or more networkdevices (such as a control device, a network-enabled personal computer,or a networked-attached storage (NAS), for example), streaming audioservices providing audio content via the Internet (e.g., the cloud), oraudio sources connected to the media playback system via a line-in inputconnection on a playback device or network devise, among otherpossibilities.

In some embodiments, audio content sources may be regularly added orremoved from a media playback system such as the media playback system100 of FIG. 1. In one example, an indexing of audio items may beperformed whenever one or more audio content sources are added, removedor updated. Indexing of audio items may involve scanning foridentifiable audio items in all folders/directory shared over a networkaccessible by playback devices in the media playback system, andgenerating or updating an audio content database containing metadata(e.g., title, artist, album, track length, among others) and otherassociated information, such as a URI or URL for each identifiable audioitem found. Other examples for managing and maintaining audio contentsources may also be possible.

The above discussions relating to playback devices, controller devices,playback zone configurations, and media content sources provide onlysome examples of operating environments within which functions andmethods described below may be implemented. Other operating environmentsand configurations of media playback systems, playback devices, andnetwork devices not explicitly described herein may also be applicableand suitable for implementation of the functions and methods.

e. Example Plurality of Networked Devices

FIG. 5 shows an example plurality of devices 500 that may be configuredto provide an audio playback experience based on voice control. Onehaving ordinary skill in the art will appreciate that the devices shownin FIG. 5 are for illustrative purposes only, and variations includingdifferent and/or additional devices may be possible. As shown, theplurality of devices 500 includes computing devices 504, 506, and 508;network microphone devices (NMDs) 512, 514, and 516; playback devices(PBDs) 532, 534, 536, and 538; and a controller device (CR) 522.

Each of the plurality of devices 500 may be network-capable devices thatcan establish communication with one or more other devices in theplurality of devices according to one or more network protocols, such asNFC, Bluetooth, Ethernet, and IEEE 802.11, among other examples, overone or more types of networks, such as wide area networks (WAN), localarea networks (LAN), and personal area networks (PAN), among otherpossibilities.

As shown, the computing devices 504, 506, and 508 may be part of a cloudnetwork 502. The cloud network 502 may include additional computingdevices. In one example, the computing devices 504, 506, and 508 may bedifferent servers. In another example, two or more of the computingdevices 504, 506, and 508 may be modules of a single server.Analogously, each of the computing device 504, 506, and 508 may includeone or more modules or servers. For ease of illustration purposesherein, each of the computing devices 504, 506, and 508 may beconfigured to perform particular functions within the cloud network 502.For instance, computing device 508 may be a source of audio content fora streaming music service.

As shown, the computing device 504 may be configured to interface withNMDs 512, 514, and 516 via communication path 542. NMDs 512, 514, and516 may be components of one or more “Smart Home” systems. In one case,NMDs 512, 514, and 516 may be physically distributed throughout ahousehold, similar to the distribution of devices shown in FIG. 1. Inanother case, two or more of the NMDs 512, 514, and 516 may bephysically positioned within relative close proximity of one another.Communication path 542 may comprise one or more types of networks, suchas a WAN including the Internet, LAN, and/or PAN, among otherpossibilities.

In one example, one or more of the NMDs 512, 514, and 516 may be devicesconfigured primarily for audio detection. In another example, one ormore of the NMDs 512, 514, and 516 may be components of devices havingvarious primary utilities. For instance, as discussed above inconnection to FIGS. 2 and 3, one or more of NMDs 512, 514, and 516 maybe the microphone(s) 220 of playback device 200 or the microphone(s) 310of network device 300. Further, in some cases, one or more of NMDs 512,514, and 516 may be the playback device 200 or network device 300. In anexample, one or more of NMDs 512, 514, and/or 516 may include multiplemicrophones arranged in a microphone array.

As shown, the computing device 506 may be configured to interface withCR 522 and PBDs 532, 534, 536, and 538 via communication path 544. Inone example, CR 522 may be a network device such as the network device200 of FIG. 2. Accordingly, CR 522 may be configured to provide thecontroller interface 400 of FIG. 4. Similarly, PBDs 532, 534, 536, and538 may be playback devices such as the playback device 300 of FIG. 3.As such, PBDs 532, 534, 536, and 538 may be physically distributedthroughout a household as shown in FIG. 1. For illustration purposes,PBDs 536 and 538 may be part of a bonded zone 530, while PBDs 532 and534 may be part of their own respective zones. As described above, thePBDs 532, 534, 536, and 538 may be dynamically bonded, grouped,unbonded, and ungrouped. Communication path 544 may comprise one or moretypes of networks, such as a WAN including the Internet, LAN, and/orPAN, among other possibilities.

In one example, as with NMDs 512, 514, and 516, CR522 and PBDs 532, 534,536, and 538 may also be components of one or more “Smart Home” systems.In one case, PBDs 532, 534, 536, and 538 may be distributed throughoutthe same household as the NMDs 512, 514, and 516. Further, as suggestedabove, one or more of PBDs 532, 534, 536, and 538 may be one or more ofNMDs 512, 514, and 516.

The NMDs 512, 514, and 516 may be part of a local area network, and thecommunication path 542 may include an access point that links the localarea network of the NMDs 512, 514, and 516 to the computing device 504over a WAN (communication path not shown). Likewise, each of the NMDs512, 514, and 516 may communicate with each other via such an accesspoint.

Similarly, CR 522 and PBDs 532, 534, 536, and 538 may be part of a localarea network and/or a local playback network as discussed in previoussections, and the communication path 544 may include an access pointthat links the local area network and/or local playback network of CR522 and PBDs 532, 534, 536, and 538 to the computing device 506 over aWAN. As such, each of the CR 522 and PBDs 532, 534, 536, and 538 mayalso communicate with each over such an access point.

In one example, communication paths 542 and 544 may comprise the sameaccess point. In an example, each of the NMDs 512, 514, and 516, CR 522,and PBDs 532, 534, 536, and 538 may access the cloud network 502 via thesame access point for a household.

As shown in FIG. 5, each of the NMDs 512, 514, and 516, CR 522, and PBDs532, 534, 536, and 538 may also directly communicate with one or more ofthe other devices via communication means 546. Communication means 546as described herein may involve one or more forms of communicationbetween the devices, according to one or more network protocols, overone or more types of networks, and/or may involve communication via oneor more other network devices. For instance, communication means 546 mayinclude one or more of for example, Bluetooth™ (IEEE 802.15), NFC,Wireless direct, and/or Proprietary wireless, among other possibilities.

In one example, CR 522 may communicate with NMD 512 over Bluetooth™, andcommunicate with PBD 534 over another local area network. In anotherexample, NMD 514 may communicate with CR 522 over another local areanetwork, and communicate with PBD 536 over Bluetooth. In a furtherexample, each of the PBDs 532, 534, 536, and 538 may communicate witheach other according to a spanning tree protocol over a local playbacknetwork, while each communicating with CR 522 over a local area network,different from the local playback network. Other examples are alsopossible.

In some cases, communication means between the NMDs 512, 514, and 516,CR 522, and PBDs 532, 534, 536, and 538 may change depending on types ofcommunication between the devices, network conditions, and/or latencydemands. For instance, communication means 546 may be used when NMD 516is first introduced to the household with the PBDs 532, 534, 536, and538. In one case, the NMD 516 may transmit identification informationcorresponding to the NMD 516 to PBD 538 via NFC, and PBD 538 may inresponse, transmit local area network information to NMD 516 via NFC (orsome other form of communication). However, once NMD 516 has beenconfigured within the household, communication means between NMD 516 andPBD 538 may change. For instance, NMD 516 may subsequently communicatewith PBD 538 via communication path 542, the cloud network 502, andcommunication path 544. In another example, the NMDs and PBDs may nevercommunicate via local communications means 546. In a further example,the NMDs and PBDs may communicate primarily via local communicationsmeans 546. Other examples are also possible.

In an illustrative example, NMDs 512, 514, and 516 may be configured toreceive voice inputs to control PBDs 532, 534, 536, and 538. Theavailable control commands may include any media playback systemcontrols previously discussed, such as playback volume control, playbacktransport controls, music source selection, and grouping, among otherpossibilities. In one instance, NMD 512 may receive a voice input tocontrol one or more of the PBDs 532, 534, 536, and 538. In response toreceiving the voice input, NMD 512 may transmit via communication path542, the voice input to computing device 504 for processing. In oneexample, the computing device 504 may convert the voice input to anequivalent text command, and parse the text command to identify acommand. Computing device 504 may then subsequently transmit the textcommand to the computing device 506. In another example, the computingdevice 504 may convert the voice input to an equivalent text command,and then subsequently transmit the text command to the computing device506. The computing device 506 may then parse the text command toidentify one or more playback commands.

For instance, if the text command is “Play ‘Track 1’ by ‘Artist 1’ from‘Streaming Service 1’ in ‘Zone 1’,” The computing device 506 mayidentify (i) a URL for “Track 1” by “Artist 1” available from “StreamingService 1,” and (ii) at least one playback device in “Zone 1.” In thisexample, the URL for “Track 1” by “Artist 1” from “Streaming Service 1”may be a URL pointing to computing device 508, and “Zone 1” may be thebonded zone 530. As such, upon identifying the URL and one or both ofPBDs 536 and 538, the computing device 506 may transmit viacommunication path 544 to one or both of PBDs 536 and 538, theidentified URL for playback. One or both of PBDs 536 and 538 mayresponsively retrieve audio content from the computing device 508according to the received URL, and begin playing “Track 1” by “Artist 1”from “Streaming Service 1.”

In yet another example, the computing device 504 may perform someprocessing to identify the relevant command or intent of the user andprovide information regarding media content relevant to the voice inputto the computing device 506. For example, the computing device 504 mayperform the speech-to-text conversion of the voice input and analyze thevoice input for a command or intent (e.g., play, pause, stop, volume up,volume down, skip, next, group, ungroup) along with other informationabout how to execute the command. The computing device 504 or thecomputing device 506 may determine what PBD commands correspond to thecommand or intent determined by the computing device 504. The command orintent determined from the voice input and/or other information relatedto executing the command may be transmitted from the computing device504 to the computing device 506. The processing on the computing device504 may be performed by an application, a module, add-on software, anintegration with the native networked microphone system softwareplatform, and/or the native networked microphone system softwareplatform.

One having ordinary skill in the art will appreciate that the above isjust one illustrative example, and that other implementations are alsopossible. In one case, operations performed by one or more of theplurality of devices 500, as described above, may be performed by one ormore other devices in the plurality of device 500. For instance, theconversion from voice input to the text command may be alternatively,partially, or wholly performed by another device or devices, such as NMD512, computing device 506, PBD 536, and/or PBD 538. Analogously, theidentification of the URL may be alternatively, partially, or whollyperformed by another device or devices, such as NMD 512, computingdevice 504, PBD 536, and/or PBD 538.

f. Example Network Microphone Device

FIG. 6 shows a function block diagram of an example network microphonedevice 600 that may be configured to be one or more of NMDs 512, 514,and 516 of FIG. 5. As shown, the network microphone device 600 includesa processor 602, memory 604, a microphone array 606, a network interface608, a user interface 610, software components 612, and speaker(s) 614.One having ordinary skill in the art will appreciate that other networkmicrophone device configurations and arrangements are also possible. Forinstance, network microphone devices may alternatively exclude thespeaker(s) 614 or have a single microphone instead of microphone array606.

The processor 602 may include one or more processors and/or controllers,which may take the form of a general or special-purpose processor orcontroller. For instance, the processing unit 602 may includemicroprocessors, microcontrollers, application-specific integratedcircuits, digital signal processors, and the like. The memory 604 may bedata storage that can be loaded with one or more of the softwarecomponents executable by the processor 602 to perform those functions.Accordingly, memory 604 may comprise one or more non-transitorycomputer-readable storage mediums, examples of which may includevolatile storage mediums such as random access memory, registers, cache,etc. and non-volatile storage mediums such as read-only memory, ahard-disk drive, a solid-state drive, flash memory, and/or anoptical-storage device, among other possibilities.

The microphone array 606 may be a plurality of microphones arranged todetect sound in the environment of the network microphone device 600.Microphone array 606 may include any type of microphone now known orlater developed such as a condenser microphone, electret condensermicrophone, or a dynamic microphone, among other possibilities. In oneexample, the microphone array may be arranged to detect audio from oneor more directions relative to the network microphone device. Themicrophone array 606 may be sensitive to a portion of a frequency range.In one example, a first subset of the microphone array 606 may besensitive to a first frequency range, while a second subset of themicrophone array may be sensitive to a second frequency range. Themicrophone array 606 may further be arranged to capture locationinformation of an audio source (e.g., voice, audible sound) and/or toassist in filtering background noise. Notably, in some embodiments themicrophone array may consist of only a single microphone, rather than aplurality of microphones.

The network interface 608 may be configured to facilitate wirelessand/or wired communication between various network devices, such as, inreference to FIG. 5, CR 522, PBDs 532-538, computing device 504-508 incloud network 502, and other network microphone devices, among otherpossibilities. As such, network interface 608 may take any suitable formfor carrying out these functions, examples of which may include anEthernet interface, a serial bus interface (e.g., FireWire, USB 2.0,etc.), a chipset and antenna adapted to facilitate wirelesscommunication, and/or any other interface that provides for wired and/orwireless communication. In one example, the network interface 608 may bebased on an industry standard (e.g., infrared, radio, wired standardsincluding IEEE 802.3, wireless standards including IEEE 802.11a,802.11b, 802.11g, 802.11n, 802.11ac, 802.15, 4G mobile communicationstandard, and so on).

The user interface 610 of the network microphone device 600 may beconfigured to facilitate user interactions with the network microphonedevice. In one example, the user interface 608 may include one or moreof physical buttons, graphical interfaces provided on touch sensitivescreen(s) and/or surface(s), among other possibilities, for a user todirectly provide input to the network microphone device 600. The userinterface 610 may further include one or more of lights and thespeaker(s) 614 to provide visual and/or audio feedback to a user. In oneexample, the network microphone device 600 may further be configured toplayback audio content via the speaker(s) 614.

III. Example Multi-Orientation Microphone Array(s) of a Media PlaybackSystem

As discussed above, embodiments described herein may involve one or moremicrophone arrays which can be trained to capture and process soundinput based on the particular orientation of a playback device.

FIGS. 7A-7B show respective perspective views of an example playbackdevice 700 in different orientations. Playback device 700 may bepositioned in different orientations by setting playback device 700 ondifferent faces or sides of playback device 700, and FIGS. 7A and 7Bshow two examples of orientations for playback device 700. FIG. 7A showsa perspective view of playback device 700 in a first orientation, andFIG. 7B shows a perspective view of playback device 700 in a secondorientation. For ease of discussion, the first orientation shown in FIG.7A where playback device 700 rests on one of its longer sides may bereferred to as a horizontal orientation, and the second orientationshown in FIG. 7B where playback device 700 rests on one of its shortersides may be referred to as a vertical orientation. The playback device700 may be set on a surface along any of its sides or faces.

Playback device 700 may have one or more microphone arrays 702 (e.g.,microphone array 702 a, microphone array 702 b, microphone array 702 c,microphone array 702 d) installed in or mounted on a housing or body ofplayback device 700. Microphone arrays 702 a-d are shown as examples ofapproximate microphone array placement integrated into a playbackdevice. More or less microphone arrays may be used and microphone arraysmay be placed along other sides or walls not shown in FIGS. 7A-7B.Further, more than one microphone array may be positioned on the sameside of a playback device.

Playback device 700 may be playback device 200 and microphone arrays 702may be microphone(s) 220. In some aspects, a microphone array 702 mayinclude a NMD (e.g., NMDs 512, 514, 516) which can be mounted on orotherwise attached to different walls or sides of playback device 700 ormay be microphone array 606. In different playback device orientations,the microphone arrays 702 will also have different orientations, anddifferent microphone arrays will be more conducive to capturing soundinput (e.g., voice input, audio input, tonal input) depending on theirindividual orientations. Each microphone array may be comprised of twoor more individual microphones positioned at different points in thearray. The microphone array 702 may have circular shape, and theindividual microphones may be distributed around a circumference of themicrophone array 702 (e.g., every x degrees between 0 to 360 degrees).For example, microphone array 702 a has individual microphones 704 a-704f.

As illustrated in FIG. 7A, playback device 700 may be placed in asubstantially or approximately horizontal orientation where a longestside of playback device 700 is parallel to a surface (e.g., table,floor) along the x-z plane. In this orientation, microphone arrays 702have various orientations. For example, a first microphone array 702 amay be on a top surface in this orientation and substantially parallelto the x-z plane while a second microphone array 702 b may be positionedsubstantially along the y-z plane and have an orientation substantiallyvertical. Microphone arrays 702 may be most sensitive to sound along aplane parallel to the microphone array. For example, microphone array702 a in the horizontal orientation (e.g., perpendicular to gravity) maybe better able to discern location of a source of sound. Microphonearray 702 a may be in a position to best detect and capture sound inputbecause of its placement on top of playback device 700.

FIG. 7B shows playback device 700 in a second orientation which may be asubstantially vertical orientation where a longest side of playbackdevice 700 is substantially perpendicular to a surface along the x-zplane and substantially parallel to the y-axis. In this orientation,microphone arrays 702 b, 702 c may be substantially horizontal andmicrophone arrays 702 a, 702 d may be substantially vertical.

Other placement positions are possible which are not shown including,for example, along the face or the back of playback device 700.Additionally, other shapes of speakers (e.g., cylindrical, triangular,irregular) may be possible providing microphone array orientations whichmay be oriented between different planes. In some aspects, themicrophone array might not be positioned to be parallel to the wall ofthe playback device nearest to it.

FIG. 8 shows an example flow diagram 800 of a method for processingsound input based on playback device orientation. Method 800 presents anembodiment of a method that can be implemented within an operatingenvironment involving, for example, the media playback system 100 ofFIG. 1, one or more of the playback device 200 of FIG. 2, one or more ofthe control device 300 of FIG. 3, one or more of the plurality ofdevices in system 500 of FIG. 5, one or more of the plurality of devicesin system 600, and playback device 700 in FIGS. 7A-7B. Method 800 mayinclude one or more operations, functions, or actions. Although theblocks in FIG. 8 are illustrated in sequential order, these blocks mayalso be performed in parallel, and/or in a different order than thosedescribed herein. Also, the various blocks may be combined into fewerblocks, divided into additional blocks, and/or removed based upon thedesired implementation.

In addition, for method 800 and other processes and methods disclosedherein, the flow diagrams show functionality and operation of onepossible implementation of present embodiments. In this regard, eachblock may represent a module, a segment, or a portion of program code,which includes one or more instructions executable by one or moreprocessors for implementing specific logical functions or steps in theprocess. The program code may be stored on any type of computer readablemedium, for example, such as a storage device including a disk or harddrive. The computer readable medium may include non-transitory computerreadable medium, for example, such as computer-readable media thatstores data for short periods of time like register memory, processorcache and Random Access Memory (RAM). The computer readable medium mayalso include non-transitory media, such as secondary or persistent longterm storage, like read only memory (ROM), optical or magnetic disks,compact-disc read only memory (CD-ROM), for example. The computerreadable media may also be any other volatile or non-volatile storagesystems. The computer readable medium may be considered a computerreadable storage medium, for example, or a tangible storage device. Inaddition, for method 800 and other processes and methods disclosedherein, each block in FIG. 8 may represent circuitry that is wired toperform the specific logical functions in the process.

At block 802, microphone training may be performed on the microphonearray(s) 702 of playback device 700. Microphone training may produce aresponse profile for the microphone array(s) of playback device 700.Microphone training may be completed during a calibration process whichmay be, for example, as part of the manufacturing process or aftermanufacturing. Microphone training may include capturing individualmicrophone responses (e.g., polar response) to test audio (e.g., tone,audio, sound, voice, noise) played from a speaker. To generate trainingresponse profile(s) for the microphone arrays 702, playback device 700may be placed in a first orientation in a perfect or nearly perfectacoustical environment (e.g., anechoic chamber) or other acousticenvironment. The test audio may be played from a speaker (e.g., speaker902) directed towards playback device 700.

A top view of an example of the calibration setup 900 is described withrespect to block 802 and shown in FIG. 9. For a first measurementcapture, speaker 902 may be directed towards the front center ofplayback device 700 which may be on axis with a center of microphonearray 702 a, and this position may be referred to as being a startingtest position and/or identified as being at the 0 degree position. Thetest audio may be played by speaker 902. The first microphone array mayreceive the test audio and measure the response of each of theindividual microphones in the array. The measured response of eachindividual microphone to the test audio may be stored as a set ofmeasured responses for a first test audio position (e.g., 0 degreeposition) and/or the measured response may be stored as a relative value(e.g., difference) between the responses of different pairs ofindividual microphones for a first test audio position (e.g., 0 degreeposition). Each response value may have a magnitude and phase component.A first set of measured responses may be stored as a matrix or tablesuch as Table 1 shown below. Microphone array 702 a may have sixindividual microphones identified as microphones 704 a, 704 b, 704 c,704 d, 704 e, and 704 f.

TABLE 1 Mic 704 a b c d e f a — — — — — — b R_(b,a) = R_(b) − R_(a) — —— — — c R_(c,a) = R_(c) − R_(a) R_(c,b) = R_(c) − R_(b) — — — — dR_(d,a) = R_(d) − R_(a) R_(d,b) = R_(d) − R_(b) R_(d,c) = R_(d) − R_(c)— — — e R_(e,a) = R_(e) − R_(a) R_(e,b) = R_(e) − R_(b) R_(e,c) = R_(e)− R_(c) R_(e,d) = R_(e) − R_(d) — — f R_(f,a) = R_(f) − R_(a) R_(f,b) =R_(f) − R_(b) R_(f,c) = R_(f) − R_(c) R_(f,d) = R_(f) − R_(d)R_(f,e = R) _(f) − R_(e) —

In Table 1, the value R_(b,a) stored in (row b, column a) may be arelative value of the response of microphone 704 b to microphone 704 ain the form of the response value measured for microphone 704 b (R_(b))minus the response value of microphone 704 a (R_(a)). This may berepeated for each microphone relative to every other microphone in themicrophone array as shown in Table 1 following the same convention. Theresponse value may be a complex number where the magnitude of theresponse may be the real portion of the value and the phase may be theimaginary portion of the value. Because the values for what would beresponse R_(a,b) in (row a, column b) may be duplicative of responseR_(b,a) for (row b column a), a response value might not be stored forthis relative response which is shown as cells having dashes (i.e.,“--”) in Table 1. The set of values in Table 1 may be associated withthe positional placement (e.g., relative angle) of the speaker 902playing the test audio.

A similar set of values may be generated for microphone array 702associated with a second speaker position by changing the positioning ofthe speaker relative to the front center of playback device 700 by agiven number of degrees while maintaining the same distance from thecenter of playback device 700 or microphone array 702. For example, asecond set of values for the first microphone array at the relativeangle of 90 degrees may be generated or obtained by moving speaker 902to the 90 degree position or rotating microphone array 702 a or playbackdevice 700 around a vertical axis (e.g., yaw) so that speaker 902 alignswith the 90 degree position of microphone array 702 a. The measurementsmay be taken in smaller increments of every x degrees (e.g., every 1degree, every 5 degrees, every 10 degrees, etc.) rotationally (e.g.,clockwise, counter-clockwise) around the playback device depending ondesired data resolution from 0 to 360 degrees around playback device700.

The measurement process in block 802 may be repeated to collect valuesfor each microphone array 702 of playback device 700 in eachorientation. For example, while in a first speaker position, responsevalues for all of the microphones of different microphone arrays may bedetermined each time test audio is played. In some instances, test audiomay be played once per angle position and individual response values forall individual microphones and all microphone arrays determined based onthe same test audio playback. Other methods of obtaining response valuesmay be used.

At or after the end of block 802, the measurements obtained through thisprocess may include response values of all of the microphone arrays formultiple orientations. The measurements may be organized in differentdata sets for different response profiles such as a collection ofmeasurements for each microphone array. For example, a first collectionof measurements may include all measurements obtained for microphonearray 702 a, and a second collection of measurements may include allmeasurements captured for microphone array 702 b.

In some aspects, the response values may be associated with each otherin the form of groups of response values determined for the samemicrophone array in two or more different playback device orientations.A group of calibration measurements may be those measurements for aparticular microphone array taken while the playback device is in aparticular orientation. For example, a first group of calibrationmeasurements may be those measurements for microphone array 702 aobtained while playback device 700 is in the horizontal orientationshown in FIG. 7A, and a second group of calibration measurements may bemeasurements obtained for microphone array 702 a while playback device700 is in the vertical orientation shown in FIG. 7B.

In another aspect, as discussed above, a first set of measurements maybe for example the measurements shown in Table 1 which can bemeasurements for the microphone generated from test audio played backfrom a first angular position for the first microphone array while inthe first playback device orientation. More particularly, the first setof measurements may be measurements for each individual microphone 704of microphone array 702 a

In some aspects, the measurements may be organized as vectors. Eachvector may correspond to one individual microphone and includemeasurements values for the individual microphone across one or morespeaker positions for the same orientation. For example, a first vectormay include values for microphone 704 a or relative values for thedifference between microphone 704 b and 704 a in the horizontalorientation for all or every measured increment between 0 to 360 degreesfor the position of speaker 902.

In some aspects, measurements may be taken for all microphone arraysfrom the same test audio. For example, the test audio can be played onceper orientation, and response values for any or all of the individualmicrophones of the microphone arrays can be gathered. The values inTable 1 could be expanded to include response values or relativeresponse values for individual microphones of more than one microphonearray. An example is shown below in Table 2 for microphone array 702 aand microphone array 702 c. Microphone 702 c array includes individualmicrophones 706 a-f. In the example Table 2 below, R_(y,z) representsresponse value of microphone y minus the response value of microphone z.For example, R_(706d, 704c)=response value of microphone 706 d—responsevalue of microphone 704 c.

TABLE 2 Mic 704a 704b 704c 704d 704e 704f 706a 706b 706c 706d 706e 706f704a — — — — — — — — — — — — 704b R_(704b,704a) — — — — — — — — — — —704c R_(704c,704a) R_(704c,704b) — — — — — — — — — — 704d R_(704d,704a)R_(704d,704b) R_(704d,704c) — — — — — — — — — 704e R_(704e,704a)R_(704e,704b) R_(704e,704c) R_(704e,704d) — — — — — — — — 704fR_(704f,704a) R_(704f,704b) R_(704f,704c) R_(704f,704d) R_(704f,704e) —— — — — — — 706a R_(706a,704a) R_(706a,704b) R_(706a,704c) R_(706a,704d)R_(706a,704e) R_(706a,704f) — — — — — — 706b R_(706b,704a) R_(706b,704b)R_(706b,704c) R_(7046b,704d) R_(706b,704e) R_(706b,704f) R_(706b,706a) —— — — — 706c R_(706c,704a) R_(706c,704b) R_(706c,704c) R_(706c,704d)R_(706c,704e) R_(706c,704f) R_(706c,706a) R_(706c,706b) — — — — 706dR_(706d,704a) R_(706d,704b) R_(706d,704c) R_(706d,704d) R_(706d,704e)R_(706d,704f) R_(706d,706a) R_(706d,706b) R_(706d,706c) — — — 706eR_(706e,704a) R_(706e,704b) R_(706e,704c) R_(706e,704d) R_(706e,704e)R_(706e,704f) R_(706e,706a) R_(706e,706b) R_(706e,706c) R_(706e,706d) —— 706f R_(706f,704a) R_(706f,704b) R_(706f,704c) R_(706f,704d)R_(706f,704e) R_(706f,704f) R_(7046,706a) R_(706f,706b) R_(706f,706c)R_(706f,706d) R_(706f,706e) —

The response values discussed above may generally represent planarmeasurement values. In other words, the values may correspond tomeasurement information gathered for two dimensions. For example, formicrophone array 702 a in the horizontal orientation shown in FIG. 7A,the measurements may represent data along a horizontal plane (e.g., x-zplane). This can be achieved by, for example, maintaining the sameheight in the placement of speaker 902 when the test audio is played.While the examples have been discussed with respect to gatheringresponse data along the horizontal plane, the data may be gathered in avertical plane rotating playback device 700 or speaker 902 vertically(e.g., roll and/or pitch) around different axes (e.g., longitudinaland/or lateral) while maintaining the same distance from the centralpoint.

In some aspects, the calibration values could be three-dimensional(e.g., spherical) by using a combination of data from different planes,for example, adding information gathered along a second plane (e.g.,plane orthogonal to the first plane) to information gathered along thefirst plane. For example, measurements may be gathered along the y-zplane (e.g., vertically) by rotating speaker 902 around the x-axis andmeasuring the response in a similar fashion as described above.

The training or measurement values discussed herein may be obtainedusing one or more devices in a system. For example, playback device 700may determine the responses of the test audio received by the microphonearrays. Playback device 700 can store the response values locally in amemory of playback device 700 and/or transmit the response values to acomputing device (e.g., server, computer, or other measurement device)for processing and/or storage.

The measured response values obtained for a given playback device modelmay be associated with the given playback device model and/or speakertype and stored in a database or server as representative values for agiven playback device model and/or microphone array model and/orindividual microphone components. For example, the same values may beused for microphone arrays or individual microphones provided from thesame supplier used in the same playback device model which may beidentified using any combination of playback device identifiers (e.g.,model number, serial number, supplier identifier) or microphoneidentifiers (e.g., model number, serial number, supplier identifier), orother calibration identifier indicating the same combination of playbackdevice identifier and microphone identifier. Using the identifiers,playback devices 700 may be pre-loaded with representative calibrationdata during manufacturing or prior to providing the playback device to auser such that the playback device is ready to adjust directional focusof its microphone arrays. In some aspects, the response values for theplayback device may be loaded or updated upon connection of the playbackdevice to a network. The values may be retrieved by or transmitted tothe playback device from a computing device over a network. In someinstances, the loading or updating of the values may occur during aplayback device setup process or other calibration process for theplayback device (e.g., environmental playback device tuning), playbackequalizer adjustment process).

At block 804, the orientation of the playback device may be determined.The orientation may be determined based on sensors (e.g., accelerometer,gyroscope, etc.) in playback device 700. Orientation of a playbackdevice may be determined as described in application Ser. No. 13/186,249filed on Jul. 19, 2011 and issued as U.S. Pat. No. 9,042,556 on May 26,2013 and application Ser. No. 14/696,041 filed on Apr. 24, 2015 andpublished as U.S. Patent Application Publication 2016/0315384 A1 on Oct.27, 2016, the contents of both are incorporated by reference in theirentirety. Orientations of the playback device may be predetermined. Forexample, the predetermined orientations may correspond to orientationsfor which training responses were measured in block 802.

At block 806, one or more microphone arrays may be determined for use tocapture or process sound input. Playback device 700 may have a singlemicrophone array in which case the single microphone array would beselected to be used in sound input capture. In some aspects, playbackdevice 700 may have more than one microphone array in which case one ormore than one microphone array may be selected.

A number of microphone arrays 702 and the particular microphone arraysthat are selected for use may depend on a variety of factors such aslocation of the microphone array 702, available processing capability,orientation of the playback device, and/or other contextual information(e.g., position of microphone in relation to the environment).

When evaluating the amount of processing capability available, playbackdevice 700 or other device in the system can base the evaluation on theprocessing power available to playback device 700 locally, to playbackdevice 700 over one or more networks (e.g., local area network, widearea network), and/or to playback device 700 via one or more computingdevices. For example, the selected microphone arrays of playback device700 may capture an initial portion of the sound input and transmit thedata representing the captured initial portion to another device (e.g.,server, control device, other playback devices) to process for locationinformation. In some aspects, playback device 700 may process the datalocally to determine location information associated with the soundinput.

The number of microphone arrays to use may be determined during aninitial setup process and/or during a playback device calibrationprocess in a playback environment (e.g., tuning of playback device basedon playback environment).

In some aspects, the microphone array that is on top of the playbackdevice may be selected. In some instances, the microphone array mostsensitive to sound in front of the playback device may be selected. Inyet more aspects, the microphone most perpendicular to gravity may beselected. A weighting or preference order of microphone arrays may begiven based on the orientation of playback device.

If playback device 700 has only a single microphone array 702 or onlyone microphone array of a plurality of microphone arrays 702 is selectedto be enabled or used for sound input capture or processing, the oneselected microphone array 702 may be enabled or activated for soundinput capture. In some aspects, adjustment of directional focus of themicrophone array(s) might not be necessary, and playback device 700 maybe ready to process sound input for execution of the command afterselecting one or more microphone arrays to enable or activate.

After the microphone array(s) to be enabled have been determined, thesystem may identify sets of response data which correspond to theselected microphone array and orientation of playback device 700. Forexample, if playback device 700 is currently oriented in the firstorientation as illustrated in FIG. 7A and microphone array 702 a isselected or activated for sound input, the system (e.g., any of deviceof FIG. 5) may identify a group of response values which correspond tomicrophone array 702 a and the current orientation (e.g., the firstorientation) of playback device 700.

At block 808, sound input may be detected by the selected microphonearray(s). For example, the selected microphone array(s) may becontinuously monitoring for the start of a voice command. A voicecommand may begin with an activation word or phrase (e.g., wake-up word,hotword) to notify the system that a user speaking the command ispreparing to speak a command for execution by the system. Uttering theactivation word can cause the system to begin processing the wordsspoken after the activation word for a voice command. Upon detecting thestart of the voice command which may be after detection of the utteranceof the activation word or include detection of the utterance of theactivation word, the selected microphone array may begin capturing(e.g., recording, streaming, processing) the sound input (e.g., voiceinput) for voice command processing.

At block 810, the location (e.g., direction in one or more dimensions,direction in one or more planes) of the source of the sound input (e.g.,voice input, audio input) may be determined. The location of the sourceof the sound input may be determined to varying degrees of precision.For example, the location may be an exact direction or an approximatedirection per microphone array or for playback device 700 or may be adirection relative to the selected microphone array(s) or playbackdevice 700. A portion or sample of the sound input may be captured andused in determining the location of the source of the sound input. Forexample, the sound input sample may be in the form of the responsepicked up by each of the individual microphones of the selectedmicrophone array. The response values may be the actual response pickedup by each of the individual microphones or may be relative valuesbetween different pairs of individual microphones of the microphonearray. These values may be calculated in a manner similar to thetraining response values obtained in block 802. For example, thereceived sample input may be processed in the same manner as the testaudio where differences between different pairs of individualmicrophones of one or more microphone arrays may be calculated. In otherwords, the received sample input may be organized into the same formatas Table 1 or Table 2 where the angle association is unknown and will bedetermined in block 810.

The response values of the sound input sample may be compared to thetraining response values to determine a direction relative to themicrophone array or playback device of the source of the input. Thecomparison may involve comparing each set of training response values todetermine which set of values corresponds to the sample input response.Because each set of training response values corresponds to an anglevalue, the direction of the sound input may correspond to the anglevalue of the set of training responses which corresponds to the soundinput. A probability function (e.g., probability distribution function(pdf), gaussian distribution) may be used to determine a most likelydirection of the sound input by determining to which training responseset or value the sample input response correlates and the correspondingangle position of the training response set or value.

In the aspect where a single microphone array is active or enabled, thesample input response values may be compared to the group of trainingresponse values associated with the one microphone array for all testposition angles to determine a most likely direction of the sound input.For example, the set of sample response values may be inputted into aprobability function such as a probability density function and comparedto the set of response values in Table 1 to determine a likelihood thatthe direction of the sound input is from the 0 degree position. Theresult of the probability function may be a probability value (e.g.,probability density function distance value) indicating the probabilitythat the sound input direction is the direction (e.g., 0 degreeposition) associated with the set of values. In some aspects, theprobability value may be a percentage where the larger the percentagethe greater the likelihood that the sound input value corresponds withthe direction associated with the set of values. In other aspects, theprobability value may be the probability density function distance valuewhere the smaller the value the greater the likelihood the sound inputvalue corresponds to the direction associated with the set of values.

The sample input response values may be compared to a subset of thegroup of values or the entire group of values producing a probabilityvalue for each of the subset or group of values. For example, trainingresponses may have been obtained for every 5 degrees between 0 to 360degrees. Such a high resolution of data might not be necessary, and thesystem may compare training responses for every 10 degrees between 0 to360. In some instances, training responses may be compared for every 20degrees, and the comparison process may be repeated for every x numberof degrees (e.g., 1 degree, 5 degrees, etc.) within the range of ±acertain number of degrees (e.g., 10 degrees, 15 degrees, etc.) of a mostlikely corresponding direction as indicated by a probability value. Inthe instance where, for example, a probability density function distancevalue represents the probability, the minimum or smallest probabilitydensity function distance value and its associated degree position maycorrespond to the most likely direction of the sound input.

As discussed herein, in some aspects, two or more microphone arrays maybe active for detecting and recording sound input. The processing fortwo or more microphone arrays may be performed for each selectedmicrophone array individually. For example, the sample input responsemay be compared to the group of response values for one microphone arrayat a time producing a set of probability values for each of the two ormore microphone arrays. The values within the set of probability valuesmay each represent the likelihood that the source of the sound input isfrom a given direction.

The set of probability values for a particular microphone array may beassigned a weight. The assigned weight may be the weight associated withthe particular microphone array which can vary depending on theorientation of the playback device. For example, in the verticalorientation shown in FIG. 7B, the values associated with microphonearray 702 c may be given a greater weight than the values associatedwith microphone array 702 a since microphone array 702 c may providemore useful information regarding a location of the audio source than amore vertical microphone.

The sample input response may be compared to the group of responsevalues for all individual microphones in the microphone arrays at thesame time. For example, the sound input values may be organized as a setof input response values with relative response values for every uniquecombination of pairs of individual microphones. The set of inputresponse values may be compared to one or more sets of training responsevalues where a set of training response values contains trainingresponse values for every unique combination of pairs of individualmicrophones.

Similar to the earlier discussion, response values for certainmicrophone arrays may be weighted more heavily by weighting thecorresponding probability value produced for that microphone array.

Similarly, the sample input response values may be compared to vectorsof training response values.

As discussed herein, a response value may have two components, amagnitude component and a phase component. The magnitude may indicatethe loudness or amplitude of the sound received by a microphone, and thephase may indicate the timing of a sound received by the microphone. Forexample, a comparison of phase information between two microphones mayindicate that one microphone received a particular sound before anothermicrophone. Phase information may be used to identify when a receivedsound is a reflection.

In the comparisons between the sample input response and trainingresponse values discussed herein, any combination of magnitude and phaseof the response values may be compared in determining the likelihoodthat a sound source is in a particular direction. For example, only themagnitudes might be compared, only the phase values might be compared,or the magnitudes and phase values may be compared.

At block 812, the directional focus of the selected microphone array(s)may be adjusted based on the location of the source determined in block810. For example, the location information can be used to informbeamforming and/or acoustic echo correction (aec) processes during thevoice input capture process to improve the quality of the voice inputcaptured.

At block 814, the sound input may be captured based on the adjusteddirectional focus of the microphone array. For example, the selectedmicrophone array(s) may be beamformed in a direction determined based onthe location determined in block 810. In some instances, an adjust mightnot be necessary. For example, the system may determine that an existingor current directional focus may be appropriate or sufficient, or thesystem may determine that directional focus might not be necessary soundinput capture.

At block 816, the captured sound input may be processed to identify anyvoice commands contained in the sound input. The captured sound inputmay be converted from speech to text, and at block 818, any commandscontained in the sound input for the media playback system may beexecuted.

Blocks 804 to 806 may be repeated each time playback device 700 or otherdevice in the system determines that the orientation and/or positioningof playback device 700 has been changed when in a media playbackenvironment (e.g., environment illustrated in FIG. 1). For example,sensors (e.g., accelerometer, gyroscope) in a playback device candetermine when there has been movement and/or a change in orientation.

Method 800 or a subset of the blocks of method 800 may be repeatedperiodically, aperiodically, and/or in response to particular eventsoccurring (e.g., tuning of playback characteristics to an environment,setup in a new environment, change in device orientation).

IV. Conclusion

The description above discloses, among other things, various examplesystems, methods, apparatus, and articles of manufacture including,among other components, firmware and/or software executed on hardware.It is understood that such examples are merely illustrative and shouldnot be considered as limiting. For example, it is contemplated that anyor all of the firmware, hardware, and/or software aspects or componentscan be embodied exclusively in hardware, exclusively in software,exclusively in firmware, or in any combination of hardware, software,and/or firmware. Accordingly, the examples provided are not the onlyway(s) to implement such systems, methods, apparatus, and/or articles ofmanufacture.

Additionally, references herein to “embodiment” means that a particularfeature, structure, or characteristic described in connection with theembodiment can be included in at least one example embodiment of aninvention. The appearances of this phrase in various places in thespecification are not necessarily all referring to the same embodiment,nor are separate or alternative embodiments mutually exclusive of otherembodiments. As such, the embodiments described herein, explicitly andimplicitly understood by one skilled in the art, can be combined withother embodiments.

The specification is presented largely in terms of illustrativeenvironments, systems, procedures, steps, logic blocks, processing, andother symbolic representations that directly or indirectly resemble theoperations of data processing devices coupled to networks. These processdescriptions and representations are typically used by those skilled inthe art to most effectively convey the substance of their work to othersskilled in the art. Numerous specific details are set forth to provide athorough understanding of the present disclosure. However, it isunderstood to those skilled in the art that certain embodiments of thepresent disclosure can be practiced without certain, specific details.In other instances, well known methods, procedures, components, andcircuitry have not been described in detail to avoid unnecessarilyobscuring aspects of the embodiments. Accordingly, the scope of thepresent disclosure is defined by the appended claims rather than theforgoing description of embodiments.

When any of the appended claims are read to cover a purely softwareand/or firmware implementation, at least one of the elements in at leastone example is hereby expressly defined to include a tangible,non-transitory medium such as a memory, DVD, CD, Blu-ray, and so on,storing the software and/or firmware.

We claim:
 1. A playback device comprising: a set of microphones; anorientation sensor; one or more processors; and tangible, non-transitorycomputer-readable medium having stored thereon instructions executableby the one or more processors to cause the playback device to performfunctions comprising: determining that the playback device is in aparticular orientation based on an indication from the orientationsensor; selecting a subset of the set of microphones for sound inputdetection based on the determination that the playback device is in theparticular orientation; detecting, via the selected subset of the set ofmicrophones, a sound input; processing the sound input detected via theselected subset of the set microphones; and executing one or morecommands contained in the processed sound input.
 2. The playback deviceof claim 1, wherein the functions further comprise: activating theselected subset of the set of microphones.
 3. The playback device ofclaim 1, wherein the selected subset of the set of microphones is afirst subset, and wherein the functions further comprise: deactivating asecond subset of the set of microphones not in the first subset.
 4. Theplayback device of claim 1, wherein the particular orientation is afirst orientation, wherein the subset of the set of microphones is afirst subset, wherein the sound input is a first sound input, andwherein the functions further comprise: determining that the playbackdevice is in a second orientation based on an indication from theorientation sensor; selecting a second subset of the set of microphonesfor sound input detection based on the determination that the playbackdevice is in the second orientation; detecting, via the selected secondsubset of the set of microphones, a second sound input; processing thesecond sound input detected via the selected second subset of the set ofmicrophones; and executing one or more commands contained in theprocessed second sound input.
 5. The playback device of claim 4, whereinthe functions further comprise: activating the selected second subset ofthe set of microphones.
 6. The playback device of claim 4, wherein thefunctions further comprise: deactivating the first subset.
 7. Theplayback device of claim 1, wherein the particular orientation is ahorizontal orientation wherein the longest side of the playback deviceis substantially parallel to a surface on which the playback device ispositioned.
 8. The playback device of claim 1, wherein the particularorientation is a vertical orientation wherein the longest side of theplayback device is substantially perpendicular to a surface on which theplayback device is positioned.
 9. The playback device of claim 1,wherein detecting, via the selected subset of the set of microphones, asound input comprises: detecting via the selected subset of microphones,a portion of the sound input; determining a location of a source of thesound input based on the particular orientation and the detected portionof the sound input; adjusting directional focus of the selected subsetof the set of microphones based on the determined location of thesource; and capturing additional portions of the sound input based onthe adjusted directional focus of the selected subset of the set ofmicrophones.
 10. Tangible, non-transitory computer-readable mediumhaving stored thereon instructions executable by the one or moreprocessors of a playback device to cause the playback device to performfunctions comprising: determining that the playback device is in aparticular orientation based on an indication from an orientation sensoron the playback device; selecting a subset of a set of microphones onthe playback device for sound input detection based on the determinationthat the playback device is in the particular orientation; detecting,via the selected subset of the set of microphones, a sound input;processing the sound input detected via the selected subset of the setmicrophones; and executing one or more commands contained in theprocessed sound input.
 11. The tangible, non-transitorycomputer-readable medium of claim 10, wherein the functions furthercomprise: activating the selected subset of the set of microphones. 12.The tangible, non-transitory computer-readable medium of claim 10,wherein the selected subset of the set of microphones is a first subset,and wherein the functions further comprise: deactivating a second subsetof the set of microphones not in the first subset.
 13. The tangible,non-transitory computer-readable medium of claim 10, wherein theparticular orientation is a first orientation, wherein the subset of theset of microphones is a first subset, wherein the sound input is a firstsound input, and wherein the functions further comprise: determiningthat the playback device is in a second orientation based on anindication from the orientation sensor; selecting a second subset of theset of microphones for sound input detection based on the determinationthat the playback device is in the second orientation; detecting, viathe selected second subset of the set of microphones, a second soundinput; processing the second sound input detected via the selectedsecond subset of the set of microphones; and executing one or morecommands contained in the processed second sound input.
 14. Thetangible, non-transitory computer-readable medium of claim 13, whereinthe functions further comprise: activating the selected second subset ofthe set of microphones.
 15. The tangible, non-transitorycomputer-readable medium of claim 13, wherein the functions furthercomprise: deactivating the first subset.
 16. A method comprising:determining, by a playback device, that the playback device is in aparticular orientation based on an indication from an orientation sensoron the playback device; selecting, by the playback device, a subset of aset of microphones on the playback device for sound input detectionbased on the determination that the playback device is in the particularorientation; detecting, by the playback device via the selected subsetof the set of microphones, a sound input; processing, by the playbackdevice, the sound input detected via the selected subset of the setmicrophones; and executing, by the playback device, one or more commandscontained in the processed sound input.
 17. The method of claim 16,further comprising: activating the selected subset of the set ofmicrophones.
 18. The method of claim 16, wherein the selected subset ofthe set of microphones is a first subset, and wherein the method furthercomprises: deactivating a second subset of the set of microphones not inthe first subset.
 19. The method of claim 16, wherein the particularorientation is a first orientation, wherein the subset of the set ofmicrophones is a first subset, wherein the sound input is a first soundinput, and wherein the method further comprises: determining that theplayback device is in a second orientation based on an indication fromthe orientation sensor; selecting a second subset of the set ofmicrophones for sound input detection based on the determination thatthe playback device is in the second orientation; detecting, via theselected second subset of the set of microphones, a second sound input;processing the second sound input detected via the selected secondsubset of the set of microphones; and executing one or more commandscontained in the processed second sound input.
 20. The method of claim19, further comprising: activating the selected second subset of the setof microphones; and deactivating the first subset.